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2  Detailed  Summary  of  Technical  Progress 

Our  work  has  focused  on  planning  and  execution  for  agents  with  imperfect 
sensors.  Our  focus  has  been  on  models  that  tightly  couple  sensing  to  action, 
but  we  have  also  studied  models  of  high-level  planning  in  simulated  dynamic 
worlds,  where  the  emphasis  is  on  projecting  and  analyzing  plans. 

Our  overall  model  of  planning  is  this:  An  agent  must  be  continously  exe¬ 
cuting  plans  in  order  to  make  progress  on  its  goals.  These  plans  are  driven  by 
sensors,  and  can  normally  cope  with  deviations  &om  expected  results  without 
intervention  from  the  planner.  When  intervention  is  required,  the  planner  starts 
from  scratch,  generating  plans  and  then  revising  them  on  the  basis  of  their  pro¬ 
jected  results.  The  projector  contains  a  probabilistic  model  of  the  world  that 
allows  the  planner  to  forecast  probable  errors. 

Our  work  on  sensor-driven  action  has  focused  on  these  areas: 

1.  Theoretical  foundations  for  set-based  decision-r  iking  algorithms. 

2.  Visual  tracking  and  vision-based  control  of  servo  systems. 

p 

'I  3.  Comparison  of  set-based  and  statistically-based  estimation. 

i 

J  Our  algorithms  for  set-based  dectsion-making  explictly  recover  the  param- 
!  eters  of  geometrical  or  physical  models  until  decision-specific  eu;curacy  criteria 
‘  have  been  met.  It  is  possible  to  supply  any  number  of  decision  criteria  that  are 
n  evaluated  in  parallel  as  parameter  recovery  is  performed.  We  have  been  able  to 
J  show  that  the  algorithm  we  use  for  evaluating  decision  criteria  is  correct  and 
'  complete  except  for  a  vanishingly  small  set  of  problems.  Correctness  means 
-  that  only  physicaUy  correct  decisions  will  be  made  as  long  as  the  input  data  are 
consistent  with  the  parametric  models  supplied  to  the  algorithm.  Completeness 
~  means  that  the  algorithm  will  terminate  on  all  inputs.  We  have  shown  that  the 
algorithm  we  use  will  terminate  on  dl  inputs  except  for  certain  boundary  cases 
that  are  typically  a  set  of  measure  zero  in  the  space  of  algorithm  inputs. 

These  results  hold  for  an  extremely  wide  variety  of  problem  settings  including 
problems  where  the  number  and  type  of  geometric  models  is  not  known  a  priori. 
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This  means  that  our  algorithm  is  effectively  a  decision  procedure  even  when  both 
segmentation  and  parameter  fitting  must  be  performed  simultaneously. 

Similar  set-based  methods  have  been  employed  by  us  for  working  in  mobile 
robot  moping  and  navigation.  At  the  same  time,  several  researchers  have 
used  classical  statistical  estimation  methods  for  the  same  purposes.  This  year 
we  have  undertaken  a  study  to  compare  statistical  estimation  methods  with 
set-based  methods  for  the  purpose  of  robot  navigation.  We  have  found  that  set- 
based  methods  typically  outperform  statistical  methods  when  the  estimation 
problem  has  low-dimension  and  is  nonlinear.  As  it  turns  out,  many  of  the 
estimation  problems  faced  in  robot  navigation  have  these  properties,  so  set- 
based  methods  would  be  expected  to  outperform  statistical  methods.  We  have 
run  both  simulated  and  real  experiments  with  a  mobile  robot  system  and  have 
verified  that  this  is  true.  These  experiments  were  done  in  the  context  of  a  map¬ 
building  system  that  uses  interval-based  methods  to  represent  its  current  model 
of  its  surroundings,  and  to  correct  the  model  as  new  information  comes  in. 

Our  final  effort  in  the  area  of  sensor-driven  plans  is  the  implementation  of 
a  system  for  real-time  tracking  in  images  using  only  standard  framegrabber 
hardware  and  standard  workstations.  This  system,  written  in  C-f-l-,  provides 
a  variety  of  tracking  “templates”  that  permit  a  user  to  define  a  collection  of 
tracked  objects  and  to  enforce  constraints  among  them.  For  example,  a  basic 
tradeable  feature  may  be  contrast  edges  in  an  image.  A  comer  can  then  be 
defined  as  the  intersection  point  of  two  contrast  edges,  the  face  of  a  rectangle 
as  a  plane  containing  four  comers,  and  a  box  as  a  collection  of  six  planes.  Each 
of  these  concepts,  a  comer,  a  face,  and  a  box,  can  be  defined  in  terms  of  the 
preceding  elements,  and  have  particular  state  variables  associated  with  them. 
So,  for  example,  a  comer  has  three  variables  (two  positions  and  one  orientation), 
a  plane  may  have  two  (a  slant  and  a  tilt)  or  more,  and  a  box  may  have  three  to  six 
as  the  user  desires.  This  tracking  system  is  currently  being  used  to  implement  a 
visual  servoing  system  for  mobile  robot  navigation  and  manipulation  of  objects. 

In  high-level  planning,  our  work  has  been  concentrated  in  these  areas,  ail  in 
the  context  of  our  transformational  planner  XFRM: 

1.  Development  and  experimental  testing  of  an  architecture  for  interleaving 
planning  with  execution. 

2.  Further  exploration  of  methods  for  representing  executable  plans  dedara- 
tively. 

3.  Optimization  and  verification  of  a  probabilistic  time  map. 

As  reported  last  year,  we  have  implemented  a  robust  interface  between  a 
planner  and  an  execution  module  that  allows  swapping  in  of  a  new  plan  at  any 
time.  We  have  ran  preliminary  experiments  in  our  “delivery  world”  that  show 
that  the  system  is  able  to  achieve  significant  improvements  in  performance  times 
simply  by  planning  simultaneously  with  execution.  Typically,  in  cases  where 
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the  planner  can  run  fast  enough  to  ‘^eat”  the  interpreter,  the  plan  it  swape  in 
embodies  speedups  over  the  default  plan  that  compensate  for  the  time  and  side 
effects  incurred  while  executing  the  default. 

TVansforming  plana  is  a  difficult  business.  To  make  it  easier,  plans  have 
to  be  represented  in  a  transparent  manner  that  enables  the  planner  to  see  the 
purposes  of  the  pieces  of  behavior  it  encapsulates.  We  have  added  more  declar¬ 
ative  constructs  for  expressing  these  purposes.  In  particular,  where  possible 
we  express  sensor  tests  using  a  BELIEF  construct  that  makes  it  explicit  which 
belie&  about  the  agent’s  environment  are  being  polled.  We  have  also  developed 
a  Prolog-like  ‘’metarlanguage”  XFRR-NL  for  expressing  transformation  rules  and 
their  associated  preconditions. 

We  have  completely  rewritten  our  probabilistic  projector  so  that  it  runs 
about  twenty  times  faster  and  embodies  more  functionality.  The  projector  now 
includes  rules  that  model  autonomous  Poisson-distributed  events.  The  rule 
(P->E  P  d  E)  specifies  that  over  any  interval  where  P  is  true,  the  expected 
time  to  the  next  occurrence  of  £  is  d.  We  have  developed  a  formal  semantics  for 
these  and  the  other  rule  types,  and  shown  that  the  program  generates  timelines 
with  the  probabilities  given  by  the  formal  semantics. 
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3  Publications,  Presentations  and  Reports 

•  Sami  Atiya  and  Gregory  D.  Hager,  “Real-Time  Vision-Based  Robot  Lo¬ 
calisation,”  IEEE  T\rana.  on  RoboUcs  and  Automation,  in  press 

•  Michael  Beets,  “Improving  Robot  Plans  During  their  Execution,”  presen¬ 
tation  at  the  Bavarian  Center  for  Knowledge-based  Systems/  University 
of  Erlangen;  the  Technical  University  of  Darmstadt;  and  the  German  Re¬ 
search  Center  for  Artificial  Intelligence,  Saarbruecken;,  Germany,  August, 
1993 

•  Sean  Engelson,  “Active  Place  Recognition  Using  Image  Signatures.”  SPIE 
1992 

•  Sean  Engelson  and  Niklas  Bertani,  “ARS  MAGNA:  The  Abstract  Robot 
Simulator  Manual.”  Yale  Computer  Science  Report  928,  1992 

•  Sean  Engelson  and  Drew  McDermott,  “Maps  Considered  As  Adaptive 
Planning  Resources”.  AAAI  Fall  Symposiiun  on  Applications  of  AI  to 
Real-World  Autonomous  Mobile  Robots,  1992 

•  Greg  Hager,  “Towards  task-directed  plruming  of  cooperating  sensors,”  pre¬ 
sentation  at  the  SPIE  Sensor  Fusion  Workshop,  1992 

•  Greg  Hager,  “Sensor  planning  for  reactive  robot  programs,”  invited  pre¬ 
sentation  at  the  Allerton  Conference  on  Control  and  Computing,  1992 

•  Greg  Hager:  “Efficient  Solution  of  Large  Systems  of  Nonlinear  Constraints 
With  Inexact  Data  and  Explicit  Termination  Criteria,”  presentation  at  the 
Conference  on  Numerical  Computation  with  Automatic  Result  Verifica¬ 
tion  in  LaFayette  Lousiana,  Feb  25-Mar  1. 

•  Gregory  Hager,  “Task-Directed  Computation  of  Qualitative  Decisions  from 
Sensor  Data,”  presentation  at  IEEE  Conf.  on  Robotics  and  Automation, 
May,  1993 
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•  Gregory  D.  Hager,  "Solving  Large  Systems  of  Nonlinear  G>nstraints  with 
Application  to  Data  Modeling,”  Interval  Computations,  in  press 

•  Gregory  D.  Hager,  "Tsak-Directed  Computation  of  Qualitative  Decisions 
from  Sensor  Data,”  submitted  to  the  IEEE  Trans,  on  Robotics  and  Au~ 
tomation 

•  Greg  Hager,  Sean  Engelson,  and  S.  Atiya.  "On  Comparing  Statistical 
and  Set-Based  Methods  in  Sensor  Data  Fusion  (with  S.  Engelson  and  S. 
Atiya).”  Proe.  International  Conference  on  Robotics  and  Automation, 
1993 

•  Drew  McDermott,  "TVansformational  planning  of  reactive  behavior.”  Yale 
Computer  Science  Report  941,  December,  1992. 

•  Drew  McDermott:  "Transformational  Planning  of  Reactive  Behavior,” 
presentation  at  the  MIT  Workshop  on  Autonomous  Underwater  Vehicles, 
Cambridge,  MA,  January  26-27, 1993 

•  Drew  McDermott,  "Planning  is  Automatic  Programming  for  Agents,”  in¬ 
vited  talk  at  the  AAAI  Spring  Symposium,  March  22-24,  1993 

•  Drew  McDermott,  “TVansformational  Planning  of  Reactive  Behavior,” 
Brandeis  University,  April  8, 1993 

•  Drew  McDermott,  panel  discussion  at  AAAI  on  "Pros  and  Cons  of  Soft¬ 
ware  Evaluation,”  July,  1993 
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4  Transitions  and  DoD  Interactions 

Prof.  McDermott  has  been  serving  on  the  Technical  Review  Board  for  the 
DARPA  Transportation  and  Scheduling  Initiative.  The  purpose  of  the  board  is 
to  provide  high-level  feedback  to  researchers  in  this  area,  using  insights  gained 
firom  past  research  on  planning  and  scheduling. 
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5  Software  and  Hardware  Prototypes 

1.  We  are  still  supporting  the  Reactive  Plan  Language  interpreter  (in  Com¬ 
mon  Lisp),  and  the  constraint  solver  (in  C  and  C+-{-). 

2.  We  are  sharing  tracking  and  mobile-robot  control  software  with  interested 
institutions. 

3.  The  probablistic  time  map  has  been  split  off  fitom  the  planner,  and  is  now 
available  via  anonymous  ftp. 
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6  Photographs,  Vugra£s  and  Videotapes 


Agent  Architecture 
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